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1 •  Introduction 

Consider  the  linear  model  given  by 

(1.1)  Yi  =  T.  +  o.Cj  for  i  =  1, . . .  ,N  , 

x.  =  x.  3  . 

1  1 

Here  {e^}  are  independently  and  identically  distributed  with  symmetric 
distribution  F,  {x^  are  (lxp)  design  constants,  3  (p*l)  is  the 
regression  vector,  and  (ck  }  expresses  the  possible  heteroscedasticity  in 
the  model.  Anscombe  (1961)  and  Bickel  (1978)  have  considered  tests  for 
the  hypothesis  of  homoscedasticity,  i.e., 

(1.2)  Hq:  =  o2  =...=  oN  . 

In  the  case  that  (1.2)  is  accepted,  we  are  in  a  standard  linear 
model  for  which  a  vast  theory  applies.  Huber  (1977)  and  others  have 
argued  (for  this  homoscedastic  case)  that  least  squares  methods  should 
not  be  used  blindly  because  the  resulting  estimates  are  sensitive  to 
outliers  and  departures  from  the  assumption  of  normal  errors.  Huber 
(1973)  and  many  others,  for  example  Krasker  (1978),  have  proposed  robust 
regression  estimates.  Huber  (1973)  and  Yohai  and  Maronna  (1979)  study 
the  asynptotic  properties  of  these  methods  as  N  -*■  °°  and  p  «> 
simultaneously.  They  argue  that  such  "many  parameter"  asymptotics  are 
important,  both  because  they  often  occur  in  practice  and  because  they 
show  how  large  a  sample  size  is  needed  for  a  given  model.  In  effect, 
their  work  formalizes  the  rule  of  thumb:  "one  should  take  about  10 
observations  for  every  parameter." 

In  contrast  to  the  homoscedastic  case,  there  has  not  been  much  work 
on  estimating  6  in  the  hetcroscedastic  case  that  (1.2)  is  rejected,  and 


to  our  knowledge  there  has  been  no  work  at  all  on  the  important  problem 
of  N  ■*  m  and  p  «>  simultaneously.  Fuller  and  Rao  (1978)  consider  the 
case  where  the  Y.  occur  in  groups  for  which  is  constant,  while  Box 
and  Hill  (1974)  and  Jobson  and  Fuller  (1980)  assume  that 

(1.3)  o.  =  [f(Ti,e0)]'1  . 

All  three  papers  consider  only  Gaussian  errors.  Box  and  Hill  suggest 
generalized  weighted  least  squares  (WLS)  wherein  the  weights  (1.3)  are 
estimated  and  WLS  applied.  Jobson  and  Fuller  consider  maximum  likelihood 
techniques  which  are  particularly  sensitive  to  non-normal  errors  as  well 
as  small  misspecifications  in  (1.3). 

For  the  case  p  fixed  and  N  -+■  »,  Ruppert  and  Carroll  (1979) 
developed  a  class  of  regression  estimates  for  the  heteroscedastic  linear 
model,  which  are  both  robust  and  do  not  depend  on  the  normal  error 
assumption.  Their  estimates  are  a  natural  generalization  of  Huber's 
Proposal  2  (1977)  and  are  defined  as  follows.  First,  assume 

(1.4)  f(r,0)  =  exp (6h (t))  ,  6(lxr)  . 

This  class  includes  the  important  special  cases 

(1.5)  f(r,0)  =  o(l+-i2)a/2  or  a  exp(or)  or  a|r|a  . 

Let  iJj  be  an  odd  function  and  x  be  an  even  function  (x(-oo)<0  ,  x(+oo)>0)  • 

A  A 

Let  13^  be  a  preliminary  estimate  of  8  and  define  9  as  any  solution  to 

1  N 

iye)  =  N']  l  x((Yi-ti)f(t.,e))h(t.)  -  o  , 
i*l 


(1.6) 
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A  /y 

where  t.  =  x.pp  .  If  an  exact  solution  to  (1.6)  does  not  exist,  define  6 
to  minimize  1 11^(0)  |.  Finally,  define  6  as  the  solution  to 

,  N  A  A 

(1.7)  N'1  >;  <H(Y.-x.  6)f(t.,0))x.  f(t.,0)  *  0  . 

i=l  1  1  lit 

We  make  the  conventions 

°0  =  (-l0R  °0  °20  — 9r0> 

(1.8) 

h(t)  =  (1  h2(T)...hr(t))  , 

with  the  parameter  chosen  so  that 

(1.9)  i:x((Yi-T.)f(Ti,e0))  =  fcX(ti)  «  0  . 

Hie  conventions  (1.8)- (1.9)  ensure  consistency  of  the  solutions.  In  the 
case  of  homoscedasticity,  the  solution  to  (1.6) -(1.7)  is  trivially 
Huber's  Proposal  2  if  the  preliminary  estimate  6^  is  also.  Ruppert  and 
Carroll  (1 1)79)  show  th.it,  in  general,  the  solution  to  (1.6)  and  (1.7)  is 
asymptotically  equivalent  to  the  "optimal"  robustified  WLS  obtainable 
when  the  (oj)  are  actually  known,  i.e., 

UNs(p.-(5))  -  Np(0  .S'1^2^)/^'^))2)  , 

N 

(1.10)  S  =  lim  N'1  l  x!  x.  f2(T.,9)  -  lim  Sy  , 

N-*»  i=l  1  1  1  N*®  N 

Np(0,V)  =  p-variate  normal,  mean  0,  covariance  V  . 

Monte -Carlo  results  show  that  the  asymptotics  are  surprisingly  accurate 
for  small  samples . 
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The  purpose  of  this  paper  is  to  generalize  the  results  of  Ruppert 
and  Carroll  to  the  case  N-*»  ,p-*°°,  under  conditions  similar  to  those 
needed  by  Bickel  (1978)  and  Yohai  and  Maronna  (1979).  In  the  next 
section  we  consider  the  estimates  of  8  when  an  appropriate  estimate  of  9 
j  ;  available,  and  in  Section  3  we  consider  the  estimation  of  0. 

2 .  The  Limit  Distribution  of  the  Regression  estimate 

A 

Assuming  that  0  is  of  the  correct  order  ((A10)  below)  and  h(*)  is 
sufficiently  smooth  ((A3)  below),  one  can  follow  the  steps  of  Yohai  and 
Maronna  (1979).  We  first  establish  the  order  of  8. 

Theorem  1.  Under  (Al)-(All)  below , 

N|P  -  6|2/pN  «  <yi)  . 

The  main  result  is  as  follows. 

Theorem  2.  Assume  (A1)-(A12)  below.  Let  be  any  symmetric  square 
root  of  T (1.10)  and  (All))  and  let  (a^)  be  a  sequence  of  (p^xl) 

vectors  with  |aN|  =  1.  Then 

N^yg-g)  i  N(0  ,  Ei^e^/O'^))2)  . 

Here  arc  the  assumptions. 

-1  N 

(A1 )  N  l  x!  x.  *  T  . 
i*l  1  1 

(A2)  hi  |  s  M  •'  ». 


» 


*■’  \ 
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(A3) 


(A4) 

(AS) 

(A6) 

(A7) 

(A8) 

CAS) 

(A10) 

(All) 

(A12) 


The  function  h(x)  and  its  first  two  derivatives  arc  continuous  and 
bounded  on  the  set  |x|  *  M  +  e  for  some  e  >  0.  Further,  < 
f(x,6)  s  Mj  on  this  set  uniformly  for  |e  -  0^1  s  e  . 

lim(pw//^/N)  max  | x .  | ^  =  0. 

N-*»  l£i<N  1 

if  is  non -decreasing  and  bounded. 

if  is  Lipschitz  and  constant  outside  an  interval. 

If  L)(u,z)  =  (if(u+z)  -  if(u))/z,  then  there  exists  b,  c,  and  d  such 
that  P(|e^|<cMj^)  >  0  and  D(u,z)  <:  d  if  |u|sc  ,  |z|sb. 

if  and  its  first  three  derivatives  are  continuous  and  bounded. 

N I Bp  -  3|2/Pn  -  0p( 1). 

N|e  -  o | 2/Pn  =  cyi). 


0  <  lim  inf  \  .  (SM)  <•  lim  sup  Xmov(SKt)  <  ,  where  X  .  (•)  and 

minv  N  *  maxv  N  ’  min 

X  (•)  are  the  minimum  and  maximum  eigenvalues  respectively. 


TnP-  0  Where  TN  -  (Bp-3)’  WqCS-6)  , 


V()  3  N  2  h  f  (-Ti,00^aNVN  xixixi  If  ^VV' 


Renarks  on  Assumptions:  Assumption  (A1)  is  a  reparameterization  device 
used  by  Muber  (1973)  and  Bickel  (1978).  Assumption  (A2) ,  which  is  also 
used  by  Bickel,  can  be  deleted  if  one  strengthens  (A3)  by  making  M  3  » , 
a  set  of  circumstances  holding  for  the  homoscedastic  case.  Assunption 
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(A3)  is  stronger  than  required  but  is  quite  convenient  anil  is  true  for 
the  first  two  special  cases  in  (1.5).  Assumptions  (A4)-(A8)  are  as  in 
Yohai  and  Maronna  (1979)  except  for  the  second  part  of  (A6) ,  which  holds 
for  those  ij;  conmonly  used  and  which  can  be  relaxed  at  the  cost  of  some 
technical  complications.  Part  (A8),  in  particular,  can  be  relaxed  if 
p^2  in  (A4)  is  strengthened  to  p^  ;  the  proof  becomes  quite  messy.  Part 
(A9)  holds,  for  example,  in  cases  of  homoscedasticity  or,  in  general,  for 
least  squares  if  litj  <  00 ,  and  (A10)  will  be  verified  in  the  next  section. 
Part  (All),  in  effect,  formalizes  (1.8)  and  follows  from  (Al)  in  the 
important  special  case  of  homoscedasticity. 

Assumption  (A12)  is  somewhat  awkward  looking  and  is  the  only 
obstacle  to  our  Theorem  2  being  a  real  generalization  of  previous  work. 
However,  (A12)  holds  in  important  special  cases,  including 
homoscedasticity,  as  the  following  shows. 

Proposition  1.  Assumption  (A12)  holds  if  any  of  the  following  obtain: 


(2.1) 

(2.2) 

(2.3) 


The  variances  are  homoscedastic. 
(p^/N)max|x. |2  +  0. 


^  A 

1*1  <  Bp  is  the  least  squares  estimate ,  and 

Ita  S,1’  WN_1 1?  xixi  '  “• 


Since  ^  is  monotone, 


HN||3  -  P|7pn  >  L2}  £  P{sup  U(y  ,  LpjJ)  >  0,  | y |  =  1}  . 


Now 


)  ~  ^Nl^  "  ^N2 ^Y ^  ’ 


where 


ani  Y  =  pn"  ^1  ^i  rCti,8))diYf(ti,6) 


N 


A  '< 


An2(y)  =  L  l  l)(zi  f(t.,0)  ,  -LdiYf(ti,0)pJ)(d.Y)2  f2(t.f0) 
i=l 


From  (A3)  and  (A4j  it  follows  that 


sup(  |  <J  •  YTfti  ,0)p^  | :  |y|  =  l,l<i<N}^0 


From  (A3)  we  find  that 


N 


AN2(y)  ^  (Ixl/Mj)  l  (diY)2  Kje.^cM^2) 


(Ld/Ml)P(|e1|scM12)  +  op(l) 


the  last  step  following  from  Y-M  and  ( A7) .  We  thus  need  only  prove 


AN1y  =  Rewrite  aniy  as 


aNiy  =  «my  +  tANrVY 


N 


«n  -  (NpN)'  l  iHz.)x  f(Tife0) 
i=l 


f  \ 


V* 
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By  (AS)  and  (All),  1-lBjJ2  «  0(1)  so  that 

sup{  IBjjyI  :  |y|  =  1}  =  0p(l)  . 

By  (A2) ,  (A3),  (A6)  and  (A8) ,  for  some  M2  >  0, 

N  a 

I  (AN1'RN)y  I  5  M2(NpN)  ”*  J  lxiyl  (|trTil  +  |9'el)  • 

By  (Al),  (A9)  and  (AID),  this  is  0^(1)  uniformly  for  |y|  =  l 
the  proof. 

Proof  of  llieorcm  2.  The  proof  is  similar  to  that  of  Theorem  : 
and  Maronna.  Ry  a  Taylor  expansion, 

0  =  ^(Yrx.B)f(ti,e))a^VN1x!qi 

=  W]  -  w2  -  w’(B-R)  +  £(B-B)'  W4(B-B)  -  iw5 


where 


qi  -  r.  =  f(T.,e0) 

w!  *  N'h  ll  ’KZiqi)qia^VN1x: 

w2  =  (L^')N'5a^VN(E-B) 

w’  =  J5,(^’(ziqi)q2  -  (E^r^V^xJx. 

W4  -  N*  £  ♦"(*iq1)^S1x!xJx.q5 
|w5l  =  0(l)|N'5s  ^  a^VN1x:(xi(B-B))3|  l  0  . 


,  couplet ing 
□ 

i.2  of  Yohai 


V 
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Vole  that  from  (A2) ,  (A3),  and  (A10) , 

(2.4)  |q.  -  r.|  =  0(|t.-T.|  +  |e-e|)  =  0(|t.-T.!  +  (p^)**)  . 

Then  (2.4),  (A8) ,  and  Theorem  1  show  that 

(2.5)  (3-3)'  W4(3-B)  -  Hh  ^  ^*(ei)r^(xi(6-3))2  a^x!  +  , 

(2.6)  1%!  =  0( l)N'?a  j£(x.(B-8))2  |q±  -  r.  |  | a^x!  |  5-  o  . 


Thus,  as  in  (3.5)  and  (3.6)  of  Yohai  and  Maronna,  we  obtain 


(2.7) 

(2.8) 


(3-3) '  W4(8-8)  -  0  , 


w}(3-3)  -  RN2  +  o  (l)  , 


RN2  *  ^  Ii(r(z.qpqf-r(ti)Tpa>}VN1xlxi(B-&) 


By  Taylor  expansions  of  ip'  and  f,  we  obtain 

w’(8-8)  =  (2BJP  +  Ee1U»"(e1))TN  +  o  (1)  , 

where  T^  is  given  in  (A12).  Because  of  (A12),  we  see  that  the  proof  is 
completed  by  showing 

f2.9)  wl  -  N"*4  *(Ei)ria^V-1x!  *  op(l)  . 

By  using  the  symmetry  of  F,  the  fact  that  \p  is  odd,  and  as  in  (3.5)  and 
(3.6)  of  Yohai  and  Maronna,  (2.9)  follows.  □ 

Proof  of  Proposition  1.  Under  (2.1), 


v* 
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a?  V  =  0  * 


verifying  (M2).  A  similar  easy  proof  follows  from  (2.2).  When  (2.3) 


holds , 


|tnI2  =  %Upn/n))°n  , 

*Vl  -  ^(aNVNlxP2  (^i^p-S))2  , 

KDj^j  *  N'1  max|x^  | 2  ^(a^V*1  Xj)2  =  OOnaxlxJ2)  , 


completing  the  proof  by  (A4) , 


3.  The  Order  of  G 


From  Theorems  1  and  2,  we  see  that  we  must  exhibit  a  sequence  of 
estimates  9  satisfying  assumption  (A10) .  We  will  show  that  any  solution 
to  (1.6)  will  work  under  the  following  assunytions. 


(Bl)  The  even  function  x  satisfies  the  same  assumptions  as  does  ij>. 


(B2)  The  statistic 


s  •  6o N_1  -  y<vN>,‘> 


(R3)  0  <  X^  =  lim  inf  ^min(Qj^)  *  li">  sup  X^CQ^  <  «  , 

Qn  =  fECjX^Ej))^1  ^  h(T.)T  h(t.)  . 


We  note  that,  when  r  is  fixed,  (B2)  holds  when  B  is  the  least 
squares  estimate,  while  Huber's  (1973)  proof  can  be  used  for  classical 
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M-estimates  if  (2.2)  holds.  Also,  (B2)  follows  from  assumption  (A9)  if 
there  is  homoscedasticity;  for  in  this  case, 

eQ  =  (log  o  0...0) 
h(x)  =  (1  h2(T)...hr(x)) 

-1  N 

(h.  =  (log  o)N  l  (t.-x.)  • 

N  i=l  1  1 

Theorem  3.  Assume  (A1)-(A9),  (A11)-(A12),  and  (B1)-(B3).  Then  (A10) 

holds. 

We  need  the  following  lemma. 

Lemma  1.  Let  {v^}  he  i.i.d.  mean-zero  hounded  random  variables;  let 
{s. }  he  uniformly  bounded  constants .  Then 

(3.1)  -  N-1  Vj.jftj-tj)  -  OpCp^N)  . 

Proof .  l’rom  (A9)  , 

AN  *  IN'1  jj  Wl1*  VV"1  '  VpftVN) 

kDn  ‘  <H1)Pn/N  • 

Proof  of  Theorem  3.  Define  H^(»)  as  in  (1.6)  and  note  that  by  (Al) , 

(A4) ,  and  (A9), 

p 

max{ |t^  -  t^|:  1  z  i  s  N}  0 

-1  N  7 

N  l  (t-T.)2  -  Op(pN/N)  . 


(3.2) 
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From  (A4) ,  (Bl)  and  Lenina  1 ,  we  thus  obtain 

1  N 

(3.3)  f^(0o)  -  N  l  xCe^hCr.)  ♦  C^Ee^'O^)  ♦  0  ( pj/N) 

ial  ** 

■  yCPl/"/1) 

from  (Bl),  (B2)  and  the  fact  that  Ex(e1)  =  0.  By  (A4),  (B1)-(B3),  (3.2) 
and  a  Taylor  expansion,  we  find  that  for  any  M2  >  0, 

(3.4)  supd^fe^AN"’5)  -  H^)  -  AQn  N'*5  | :  |a|  £  M2P^  }  5  0  . 

Now  fix  e  >  0,  L  >  0.  Choose  y,Nq,D^  such  that  N  2  NQ  implies 

PiN^I^I  *  D^}  <  e/3 

m~h\ $  x(ei)h(Ti)|  *  y)  <  e/3  . 

Define  M2  by 

L  =  \»V2  -  ^  -  d0  . 

Choose  Nj  i  Nq  so  that  N  *  Nj  implies 

WV  2  V* 

and 

P{left  side  of  (3.4)  2  y/2}  <  e/3  . 

Then  with  probability  at  least  1-e,  N  a  Nj  and  |A|  ■  M2P^  imply 


V* 
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(3.5)  (N/p^  AH^+AN'*5)  *  LpJ*  . 

Also  note  that  AH^CGp+sAN  ** )  is  increasing  as  a  function  of  s;  so  that 

\t 

if  | A |  £  M2pJ^ ,  then  as  in  JureckovA's  (1977)  proof,  using  (3.5), 

(3.6)  (N/pN)Ji  A^CO^AN"**  )  5  |A|Lpjj  . 

Prom  (3.6),  we  have  with  probability  at  least  1-c, 

infCp^lHj^e)!  :  N*5 1 G  -  80|  a:  M2pJj} 

*  inf{|A^(0o+AN^)|/|A|pg:  fA|  a  } 

a  L  . 

Since  L  is  arbitrary,  the  proof  is  complete.  Q 

Remark:  Throughout  we  have  taken  r  fixed;  r  is  the  dimension  of  0Q  and 

h(T).  This  seems  a  perfectly  reasonable  assumption,  as  one  is  unlikely 

to  construct  a  complicated  function  for  o^,  and  in  fact,  r  s  3  will 

suffice  for  most  applications.  However,  the  proofs  can  be  forced  through 
u 

when  r  =  rN  =  O(pj^).  One  must  assume  that  the  bounds  in  (A3)  are 
uniform  in  the  components  of  h(*)»  and  the  only  stumbling  block  is  the 
assumption  (B2).  When  is  the  least  squares  estimate  and  Ee^  <  ~,  it 
turns  out  that  (R2)  holds  whenever 


(Nl'J)'1  IhjMcjl  -  0(1)  . 
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